Search AI Products and News

AI security company Adversa AI has released a shocking report stating that Elon Musk's startup xAI's newly launched Grok3 model has significant vulnerabilities in terms of cybersecurity. Adversa's research team found that this latest AI model is susceptible to 'simple jailbreak attacks', potentially allowing malicious actors to access sensitive information such as 'how to groom children, handle corpses, extract DMT, and make bombs'. Worse still, Adversa

Anthropic has announced an expansion of its vulnerability reward program aimed at testing a 'next-generation AI safety mitigation system,' primarily focusing on identifying and defending against 'universal jailbreak attacks.' Special attention is given to high-risk areas, including CBRN defense and cybersecurity. Participants will have the opportunity to engage with the new safety system ahead of time, identifying vulnerabilities or bypassing security measures, with rewards of up to $15,000. This initiative aims to enhance the safety of AI systems by attracting security researchers to collaboratively discover and fix potential threats, setting a benchmark for safety in the AI industry.

OpenAI's DALL-E 3, an image-generating AI technology, has been found to be easily influenced by jailbreak prompts, resulting in the generation of photos depicting children smoking. The jailbreak technique was discovered by LinkedIn user Peter Gostev, who shared how to successfully bypass the AI's restrictions to create images related to children smoking. Although the jailbreak prompts are quite complex, this incident serves as a reminder that even the wealthiest companies in Silicon Valley struggle to establish comprehensive protective measures for their AI systems.